Design And Implementation Of An Island-Based File System

Reliability, availability and scalability are major concerns in the design of distributed file systems. We have built an island-based file system (IFS) called Archipelago to solve these problems by failure isolation and low-cost consistency maintenance. The building blocks of IFS are smaller self-contained file servers called islands. The main idea underlying island-based design is the one-island principle: as many operations as possible should involve exactly one island. The oneisland principle improves partial reliability and availability because each island can function independently of other islands’ failures. It allows IFS to scale efficiently with the system and workload sizes because consistency across islands can be maintained at a low cost. The data distribution strategies in existing file systems cannot satisfy the one-island principle without sacrificing load balance and scalability. We designed a new strategy in which data is distributed to islands at directory granularity by hashing the pathnames of directories. Certain metadata is replicated across islands in such a way that islands are self-contained and the cost for maintaining consistency across replicas is minimized. We evaluated the data distribution strategy in IFS by statistical analysis of the access patterns and contents of existing file systems in use. We studied partial availability, load balance, replication cost and consistency cost in web access logs, UNIX file system call traces, snapshots of file system contents, and Windows NT file access traces. The results confirmed the assumptions we made in the design. In addition, we compared data loss of IFS and typical non-IFS in case of partial failures in analytic models. The IFS model has a significantly lower data loss ratio than non-IFS, at the cost of replicating a small amount of metadata. We designed three protocols in Archipelago, the rebalance, consistency and recovery protocols, to make the island-based design a viable solution. We have implemented Archipelago on a cluster of PCs running Windows NT 4.0 connected by Ethernet. The consistency and recovery protocols are tested with randomized failure injections. The performance measured in micro benchmarks and operation mixes shows little overhead of the consistency protocol on oneisland operations; in one case, the speedup with 16 islands achieves 98.3% efficiency. A trace-driven study of the online reconfiguration of a web server running on Archipelago shows that data migration in the rebalance protocol is made transparent to the web server and imposes a performance penalty of only 4.5%.